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determine whether a student knows basic information about the world 
and whether he sees himself as an integral part of chat world. The 
evaluator is concerned with the quality of curriculum, teaching 
methods and materials, and with the actual content the student has 
learned. Both cognitive and affective domains must be evaluated to 
ensure that the student knows and applies relevant facts. The first 
step is for the evaluator to set up a logical framework which defines 
the goals of the course and the methods of evaluation. Suggested 
methods include interviews, research projects, and objective and 
essay tests. Interest scales, semantic differential scales, and 
Likert scales, which determine student attitudes, are also 
recommended. Directions for the construction and use of a logical 
framework, testing models, and a glossary of terms relevant to 
education and testing are included in appendices. A bibliography 
provides sources on both evaluation and global studies. (KC) 
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DOES IT WORK? 



Evaluation Guide 1 ines 
For Global Studies Teachers 



PREFACE 



"Does It Work?" is the latest step on a continuing path that has 
has been taken by the Global Development Studies program, since 
its inception in 1969, towards helping teachers and students 
struggling with understanding the world in which they live and 
work. The GDS program came from the conviction that if students 
jtfe pe - to-^dexuU e£XacX±v:eJj^w±t±^an.. ,±iicx&ja.sjLn^ n t 

world as adults, they must develop a knowledge and understanding 
of it as part of their general education. First under the di- 
rection of the Management Institute for National Development 
(MIND) in New York City, the rationale and principles behind 
teaching global development were developed in a series of re- 
search papers from 1970 to 1972. These culminated in the produc 
tion of Global Development Studies Model Curriculum , issued in 
1973, setting forth a model tor curriculum development in glo- 
bal studies for grades 11-14. This work outlined procedures 
for teaching the theories and principles behind global develop- 
ment through a model course. It was followed, in 1974, by World 
Food Supply: A Global Development Studies Case Study , which out- 
lined the basic elements of this global problem. This project 
was designed for use in conjunction with a course in global stud 
ies, such as that set up by the Model Curriculum, to tie the 
theoretical down to ! real world' conditions. It was revised, 
expanded, and reissued in 1975 and 1976. The next areas of con- 
centration were in teaching methodology and administrative sup- 
port. These were (and are) carried out through a series of sem- 
inars, conferences, and workshops, at both college and school 
levels and with both international and regional participation. 

In 1976, the GDS program, having gained such momentum since its 
inception as a small part of MIND's work, had reached a point 
of maturity and reliability where its work could be most effec- 
tively carried on as a separate institution. The Global Develop 
ment Studies Institute was established in April 1976, and moved 
to its present quarters in Madison, New Jersey in July of that 
year . 

Since 1969, more and more schools and teachers have recognized 
the need for teaching their students about global realities. 
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All sorts of courses and approaches have been used, including 
those based on and using GDS materials* The most natural ques- 
tion now is — "Does it work?". 

This paper has been prepared to help the individual teacher — 
in high schools and colleges/universities deal with that 
question in his own classroom. It examines basic approaches and 
and techniques of evaluation for the teacher to use as best fits 
his particular situation. Set up to investigate the Purpose and 
the Process of evaluation, the paper also includes Appendices of 
specific examples of 'how to*. It is easy, too, in such inves- 
tigations, to get involved in the professional jargon that can 
be murky in definition "affective", "cognitive", "summative", 
"formative". Indeed, different professionals in the field often 
use the same terms for different meanings. To help the reader 
with this problem, a Glossary is included after the Appendices, 
to indicate what this author is saying in this report. 

As always with GDS materials, this paper is not intended to be 

■frke— kfiG ^ a 1 1 and— e-nd-^a-14 —o4-~ t- h e— s u kj-eefc~ roa-fe t -e- r - * R-a~t4ve-r-; — i~t — i-s — — 

intended to set up guidelines and models for teachers struggling 
with an area of education that demands innovation and individual 
approaches , 

Comments, suggestions, and criticisms are welcome, as the art of 
evaluation is, by no means, perfected. The author, in addition, 
wishes to express her appreciation to readers of an earlier draft 
of this paper for their helpful assistance. Their advice has 
been incorporated into the study to the extent possible. 



J. Carlisle Spivey 
Vice-President 



May 1977 



DOES IT WORK? 

Evaluat ion Guide 1 ines 
For Global Studies Teachers 



INTRODUCTION 

After explaining the why and how of global studies, the most 
natural question is M Does it work? M . Are perspectives being 
broadened; does the student know basic information about the 
world; does he see himself as part of that world? If not, 
why not? Should the teacher be doing more of the same, change, 
quit trying altogether? Do the course goals make sense at all? 
These types of questions must be confronted if global studies 
courses are to be fully accepted in our schools. To do this 
means undertaking the process of evaluation. 

Most, g lobal ft ducatloi^jLodajL^is^-under taken in the individual, 
class, or, at best, in a small group of classes. Grade-wide 
and school-wide programs are, indeed, the exception. Profes- 
sional evaluators work with enormous groups compared with the 
average class in global education. This means that, in most 
cases, it must be left to the individual teacher or program 
director to evaluate his own course. Program evaluation, how- 
ever, is often the haziest of teaching techniques within the 
instructor's professional realm. * 

This paper is designed to provide guidelines to help the indi- 
vidual teacher interested in evaluating his program but who 
finds himself in complete limbo as far as "how to" is involved. 
Aimed at the high school and college/university levels, it is 
not meant to be the definitive study on evaluation theory and 
technique. The purpose, rather, is to help the struggling teacher 
by providing basic steps to a process that fits his own situation. 

Evaluation procedures should not be equated with giving grades. 
Grading is a method used in education for rating students against 
a basic pre-set standard and one another. It has very little 
to do with those aspects of teaching with which the evaluator 
is concerned: quality of curriculum, teaching methods and mater- 
ials , and the actual content that the student has learned. 



EDUCATIONAL EVALUATION: THE PURPOSE 

The broad purpose of American education is to enable the indi- 
vidual to live effectively in our society. The broad purpose 
of the evaluation process within education is to ascertain if 

- 1 - 
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the educational process has been successful. In so doing, it 
includes testing the course 1 s effect on students and the effec- 
tiveness of the program, the curriculum, the materials, and the 
teacher, as compared with the stated objectives of the course. 
Often this process leads to the reformulation of goals and ob- 
jectives, thereby improving the whole program. 

At present, the society that most educational systems prepare 
their students for is an isolated national/local one. Yet, 
every day our world becomes more and more interdependent and 
our society more global. The goal of global education is to 
give students the ability to live and work effectively in this 
increasingly global and interdependent society. 

Global education comes in many forms, under many names glo- 
bal development studies, world order studies, international edu- 
cation, to name only a few. A few objectives, however, are 
generally universal to all: 

_ — — . M -fc e -" b-y^ark— dow-n— t-he—e-tfento ee-n-fc-r -i<~~^e-fah:ey - J d ie l^-t-o-my— ~ - 

of viewing the world 

- to discover that f yes/no f , ! right/wrong 1 answers 
have little, if any, connection with the f real f 
world 

- the development of a 'global perspective 1 , recog- 
nizing that the individual , his family, town , 
state, country are integral factors in world-wide 
situations (such as world food supply) 

- to develop knowledge and understanding of uni- 
versal factors affecting the global society 
economics, population growth, geography, etc. 

These objectives require delving into both the cognitive and 
affective domains. The cognitive aspect of a course entails 
specific knowledge which can be tested for: facts, figures, 
names, dates, places, concepts. The affective aspect infers 
translation of such data into personal interpretation. The cog- 
nitive approach provides the knowledge for understanding how 
plants grow and provide nutrition for daily activities, and that 
approximately 460 million people in the world are under- or mal- 
nourished. The affective approach provides the ins ight that the 
student is part of an interdependent world, and therefore a 
world problem is his problem. Together, these two aspects join 
to give the student the ability to adequately confront opinion 
and action concerning the world food problem; a problem with 
* which he will have to deal in the adult world. In such 
a way, combining the two aspects of learning encourages students 
to use cognition to form individual opinions. 

Cognitive testing has long been an accepted practice in the 
field of education. Affective testing, however, is a newer 
area, still very much under experimentation. Objections to 
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testing affective learning usually focus on four complaints: 

- to cons id er affective testing is to accept affec- 
tive teaching which is tantamount to brainwashing 

- affective change evolves over too long a period of 
time to test for reliable results after merely the 
time period of one course 

- the affective part of a person f s life is private 
and, therefore, to undertake such testing, is an 
invasion of privacy 

- affective knowledge concerns values and it is 
not proper to afix grades to values. 

To advance any one or any combination of these objections is 
to misunderstand the purpose of both education and evaluation. 
In answer to the first general objection: education involves not 
only learning facts - - cognition , knowledge but also how to 
use these facts to form valid opinions. This includes knowing 
how and when to question 'givens', how to weigh different in- 
,famation and^ how to form and Jest „ 

hypotheses, and how to draw conclusions using the student's 
own beliefs and values. It also means learning how to re-examine 
one's conclusions when new information is available. These 
skills are part of the learning process. Rather than suggesting 
'brainwashing', they guard against that threat by enabling the 
student to form his own ideas. 

As to the second objection: significant results of education 
occur in both the short and long term. To this extent, it is 
true that all changes effected by global education cannot be 
evaluated the day the course concludes. The same is true of 
traditional courses such as literature. Some valuable shorter 
term changes in a reading course, for example, can be evaluated 
by testing such factors as comprehension, speed, vocabulary, 
and difficulty of text. But the objectives of teaching reading 
go beyond the mechanics of translating cryptics into language. 
It also includes teaching the 'values' of reading that it is 
a source of knowledge and of entertainment. It is this affec- 
tive aspect that gives reading its greatest value. This recogni- 
tion may not appear to the student in totality, or even at all, 
at the conclusion of one course in reading. It may take practice 
with the newly acquired skill to evaluate fully the overall ef- 
fect of the course. But this does not mean that evaluation 
will not reveal valuable information about the course, and its 
teaching, at its end. There is still much knowledge to be gained 
from testing procedures, though change may continue after the 
testing process. And so it is, too, with global studies eval- 
uation may not reveal all possible results, both short and long 
term, to be acquired from the course; but it will give signifi- 
cant indications of the degree of the course's success in reach- 
ing its objectives. 
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Third, the purpose of affective testing is not to pry into a 
student's most private thoughts and values. It is concerned with 
finding out if, and to what degree, the student has acquired 
the understanding needed to make *and question his own decisions 
and decide his own values. In this way, there is no threat of 
invasion of privacy. 

Finally, when the teacher undertakes course evaluation, grades 
have no place and should be ignored. The purpose is to gather 
information to weigh the worth of the course design, materials 
and methods, and its teaching in reference to the course goal. 
This is separate from comparing student achievement with a pre- 
set standard or with that of other students. There is never a 
question of ! grading values 1 . 

Just as there are two types of domains for testing cognitive 
and affective there are also two foci — formative and sum- 
mative. The two differ in purpose, time factor, and level of 
generalization. Summative testing is designed to observe over- 

-ft-H — r-f^-vi4-f~^aiL~^ s e or .siih^jtarLtial parts of it. 

Such evaluative techniques are used to test teacher effective,- 
ness, to ascertain if students have grasped the overall subject, 
and to compare curricula effectiveness. Formative evaluation 
is used during the teaching process rather than at the end. Its 
purpose is to observe curriculum construction and adaptation, 
teaching, or learning, in order to improve any one or all of them. 
This aspect of testing involves independent units of a course, 
rather than overall results of a program. Both summative and 
formative testing are important to the teacher f s evaluation pro- 
cess: to see if a course 'works 1 and why. 



EDUCATIONAL EVALUATION: THE PROCESS 

The teacher who undertakes an individual evaluation of his glo- 
bal studies course has several advantages over the 1 prof es s ional 1 
evaluator working with enormous groups: 

- he knows the nuances of his test group 

- he can be more flexible with time and method 

- he has greater insight into the situation and 
more freedom in applying his gained knowledge 
for change and improvement. 

His biggest handicap is lack of a large data bank which comes 
with sizable groups. Testing on a large scale provides plenty 
of generalizations in handy, easy-to-use numerical form that 
provides 'proof for the evaluator's results. The individual 
teacher's 'proof' rests on knowing his students and on the evi- 
dence of actual events: citing responses and changes in attitudes, 
interests, and behavior. The best aid he can have in this under- 
taking is the cooperation of his class. Students are understand- 
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ably wary of testing, especially at the high school level where 
college acceptance is so dependent on grading, class standing, 
and test scores. It must be clear to students that it is not 
the success or quality of their performance that is being judged, 
but that of the curriculum, teaching methods, and materials. 

The first step in the evaluation process is to set up the frame- 
work. Without conceptual control over the complex variety of 
events involved in this type of task, the teacher will be swamped 
by a bewildering mass of information. A succinct design for the 
project will serve to plant the teacher T s work firmly in reality 
and to provide a practical system for 'reading 1 the results, 
making the study a useful, rather than merely an academic , pro - 
cedure. The framework is not rvn evaluation technique itself; 
rather, its job is to set the stage for the evaluation process. 
As such, it calls on the evaluator/ teacher to: 

- define the goals of his course 

- outline the evaluation procedure and the 

_ p.u-.pp-as~e~o^ 

- plan for methods and materials (input) and 

for expectation of achieved learning and under- 
standing (output) 

- specify evidence (indicators) chosen to verify 
output 

- state assumptions about students , methods , 
materials, and the teacher, that are made in 
choosing indicators and planning for output. 

A useful framework to adapt is one designed by the Department of 
State's Agency for International Development: the Project Design 
Summary Logical Framework, used for evaluating development as- 
sistance programs. More information on the 1 logical framework 1 
and its use can be found in Appendix A (page 15), 

In using the 'logical framework', or any other outline the 
teacher chooses, several criteria are necessary for best results: 

- the tester should be as obj ective as possible 

- goals, assumptions, and conclusions must be 
operationally useful 

- reporting must be done in a form readily under- 
standable to those using the results 

- the tester should not just be measuring pro- 
gress, but also questioning the premise on 
which the project is based. 

This may seem a tiresome and overly complex chore, but it is 




■•■Office of Program Evaluation, U.S. Agency for International 
Development, Evaluation Handbook, second edition (Washington, 
D.C.: U.S. Department of State, 1974), p. 35. 
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a necessary one. It has the value of enabling the evaluator 
to recognize his own assumptions and weigh them against what 
he feels are basic standards for an effective course. It also 
allows those who are interested in his results to understand 
the process and reasoning behind them. Finally, it sets up a 
schedule and work-plan to simplify the task facing the evalua- 
tor . 

Within this framework, basic questions need to be answered: 

- WHY were the specific course objectives chosen 

- WHAT is to be learned 

- WHO is involved with the procedure 

- HOW 7 is the study being done. 

The answers to these questions are critical to the teacher f s 
understanding of his evaluation as well as for outsiders who 
will try to make use of it. The questions are often inter- 
related and the answers will depend on the individual circum- 
- s ta nc e s 1 of- ettt^r- i ^4lrtt^'t4- en ' 1 ; ~~ — — ■ — — — 



Question 1: WHY ? 

This question requires the • evaluator to define the purpose of 
of the course in concrete terms. Some examples of indicators 
of course goals for students might be: 

- to understand the continuing interrelationship of 
of major global issues 

- to include historical, political, social, cul- 
tural, and economic factors and their inter- 
actions when discussing global issues 

- to defer from assigning 'right/wrong' labels 
when discussing global issues. Instead, the 
emphasis should be on the fluctuating factors 
involved and the varying cultural viewpoints 
towards them. 

Course design and evaluation are integral. 2 # With implementa- 
tion, they make up a cycle for educational improvement and inno- 
vation. Therefore, at the same time that the evaluator is out- 
lining his course goal, he will also want to underline the pur- 
pose of his evaluation process. Evaluation goals might be: 

- to determine the extent to which course goals 
are being met 

- to determine the extent to which materials 
used assist or facilitate meeting course goals 



2 0ffice of Development Program: Review and Evaluation, Proj ect 
Evaluation Guidelines (Washington, D.C.: Agency for International 
Development , U.S. Department of State, 1974), p. 1. 
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- to determine areas of needed improvement . 

The most important consideration is that these goals be stated 
in terms that are both operationally useful and readily under- 
standable . 

Question 2: WHAT? 

Having defined the goals, the next step is to identify the 
various objectives that will lead to achieving them. In other 
words: WHAT is to be learned — by the teacher as well as by 
the students? This question involves citing and describing the 
various objectives and their levels of priority that make up 
the overall goal of the course. These will depend on the educa- 
tional philosophy of both the school and the teacher. This 
question involves the skills and information necessary for 
reaching the stated goal, and the indications of achievement of 
these objectives. 

— 

outline of the course and a description of the materials and 
methods (inputs), evidence of assumed results (indicators of 
outputs), assumptions upon which the validity of these indi- 
cators are based, and hypotheses drawn as to the causative 
links between inputs, outputs, objectives, and goals (See 
Appendix B for example) . 

Finally, the teacher must question not only the content involved, 
but whether the objectives he has set up are both possible to 
achieve and desirable to the goal. As with goal statements, 
objectives must be described in operative terms that can be 
easily understood by 'outsiders 1 . 

Having drawn together this information, the evaluator will be 
able to see if, indeed, his inputs help produce the desired out- 
put which leads to achieving the stated objectives on the path 
to reaching the course goal. 

Question 3: WHO? 

The question 'who 1 includes n who is the test group", "who wants 
to know", and n who is the teacher". The first calls for rele- 
vant descriptions of the class, including: 

- what is the group's 'starting point' for the course 
course 

- what are their attitudes toward certain global 
issues 

- what is their orientation, both in and out of 
school, for the course 

- what skills do they bring with them to the 
course 
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- what are their reasons for taking the course 

- what are their number, age," etc. 

The teacher, wishing to answer ! who ? as far as his class is in- 
volved, will doubtless wish to do a pre-test of his students to 
indicate better where they are as far as his course is concerned. 
This form of testing, as well as other evaluation techniques, 
is discussed in the final 'HOW question section. 

Answering the question "who wants to know M will help the teacher 
focus his evaluation goals and course objectives, as well as 
the language of this final report of results. Is the testing 
done for his own benefit so that he can improve the course; is 
it done to convince the school board that he should be allowed 
to teach the course again; is it done for department heads to 
convince them that the orientation of other courses should con- 
tain skills that will prepare students for his course? 

Finally, the question "who is the teacher" must be answered. 

tfr a-S "'" h e had p r io n: — eirpv r i e n c e w r t h — t h e " cou r s e? — Wfttrt — t r ain i ng ha s 

he had? In what discipline of the course is he the weakest? 
What impact does this weakness have on the course? Wh at has the 
impact of the course been on his own thinking about the sub- 
ject? A History teacher, for example, who undertakes a course 
in global studies, may find that his own ignorance of biological 
systems hampers explanation of water pollution as a world prob- 
lem. The recognition of this weakness may lead him to plan in- 
terdisciplinary approaches with the help of a colleague in the 
Science department . 

Question 4: HOW? 

The question of 'how to ? is, of course, of greatest concern to 
the teacher about to actively undertake the evaluation process. 
This is especially true for the generally unfamiliar area of 
affective testing. Following are descriptions and explanations 
of various evaluative techniques that have been found useful 
by teachers struggling with similar evaluation problems. Examples 
of many of these testing techniques may be found in Appendix B. 
Teachers will want to use these general discussions and specific 
examples as guidelines and models to be used in the manner best 
suited to individual and specific situations. 

The evaluator of a global studies course is looking for evidence 
of the development of four types of knowledge: 

- concrete facts 

- knowledge comprehension: interpreting facts 

- an increased global perspective for under- 
standing issues 

- an understanding that education is not only 
memorization of facts, but a recognition of^ 
relationships and a questioning of concepts. 
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Evaluation techniques come in myriad 'packages' in structured 
or unstructured settings, among them: observation and testing; 
individually or in groups; written or oral; question/answer 
or discussion. 

The reader Kill note that many of the following techniques are 
familiar to the teacher as traditional tests for grading pur - 
poses. He should always remember, however, that the objectives 
behind the evaluation procedure and those for testing are quite 
different , 

Interviews 

A particularly effective method for the teacher undertaking 
evaluation of a single class or small group is the interview, 
either structured or unstructured. In the structured setting, 
the interviewer prepares a sequence of specific questions with 
fixed wording. His role is directive: to present the question 
and record the student's answer. The only deviations allowed 
are to clarify any misunderstandings related to the questions. 
The teacher's role in the unstructured interview is non-direc- 
tive. He has a limited number of key questions about pertinent 
topics to achieve the greatest desired analytic information 
from the conversation,^ The interviewer's role is to listen, 
rather than to lead the convers t aion , and to probe for ideas 
not immediately attributable to the key question. 

The unstructured setting has several advantages over the struc- 
tured: it affords broadei and deeper information through personal 
expressions; it can suggest ideas for teaching and writing; it 
can test the validity of structured testing information,^ In 
short, it can put meat on the skeletal structured approach. 
The major disadvantage of the unstructured approach is tnat it 
is quite time-consuming. Further, while the atmosphere of the 
unstructured interview is much more relaxed, the interviewer 
must be careful not to lead or otherwise influence the respon- 
dent's answers. 

Research 

Research projects can provide information about the student's 
ability to create, interpret, and apply knowledge. Because 
of the time factor involved in an entire research project, 
however, this process might be limited in interpreting change 
in skil 1 s due to the course's effect. Asking the student to 
describe hoitf he would go about planning and researching a parti- 
cular topic or project can provide similar insights with a time 
advantage. This is particularly true if the problem presented 



5 Ibid , , p, 237, 
4 Ibid.. 
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to the student requires the use of previously learned material 
in a never-before experienced capacity. For example, the stu- 
dent, having studied the factors involved in the increase in 
the price of oil world-wide, may be asked how he would investi- 
gate the increase in the price of coffee. Although he has not 
studied Brazilian Development, there are comparable relationships 
upon which he may draw, thereby demonstrating his grasp of certain 
concepts of development in general such as effects of tariffs 
and supply/demand relationships. Such projects can also give 
the teacher an indication of the student f s awareness of the 
necessity to know the background, purpose, and slant of input 
material . 

Written Tests 

Four types of written testing are very useful for evaluating 
global studies courses: 

- objective and essay questions measuring know- 
ledge and its application 

- Interest Scales measuring degrees of student 
interest in the various areas of the course 

- Semantic Differential scales measuring stu- 
dent attitudes 

- Likert Scales measuring student attitudes. 

- Objective and Essay Questions: 

1 familiar objective and essay questions are those best suited 
f f> r testing cognitive knowledge and the student f s ability to 
a, ply this knowledge to problems with which he is not familiar. 
< jective questions need not simply be Recall 1 tests. Ques- 
tions in reference to described problems, graphs, or pictures 
\ 11 elicit information about application skills, as well. Care 
should be taken in presenting essay questions to insure obtaining 
the type of analytical information that the teacher desires. 
It is better, for example, to ask the student .to "compare and 
contrast 11 rather than to M discuss n or "write about" a topic. 5 
Caution also should be taken to insure that the evaluator does 
not mistake a student 1 s ability (or lack of ability) to ex- 
press himself well with his knowledge and ability to apply in- 
formation . 

- Interest Scales : 

This technique is designed to measure student interest in the 
subjects of the course. Simple in construction, it is made up 
of four options which represent varying degrees of interest: 



^Stadsklev, Handbook of Simulation Gaming in Social Education , 
Part 1: Textbook (University, Alabama: Institute of Higher Edu- 
cation, 1974) , p. 113. 
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A. Dull 

B. Generally uninteresting 

C. Generally interesting 

D. Very interesting. V 

These are to be used by students to rate certain topics with 
the course. For example, the class is asked to rate the fol 
as to how interesting they find them: 

1 . World Food Supply 

2 . Braz il ian Development 

3. Study of Comparative Economies. 

Points of comparison can be assessed by including topics such 
as sports — other than those included in the course material. 

To 'read' the results of an Interest Scale, give a weighted 
value to each of the four options offered: for example, starting 
with four points for 'very interesting', ranging to one point 
for ! dull f . In this way, a class mean can be established. This 
information can be useful when compared with successive Interest 
Scale results given during the teaching of the course. The 
teacher can follow the effect of the course and note where ex- 
tra emphasis or change in approach is desired. 

- Semantic Differential: 

This technique, another designed to measure student attitudes, 
focuses on the images and meanings that students ascribe to 
certain ideas or objects. The evaluator sets up a series of 
descriptive bipolar adjectives or adjective phrases, making sure 
sure that the exptremes lie along the same continuum: M stupid/ 
smart" , " import ant/ trivial" , "energetic/ lazy" . Students are 
asked to choose one of seven points between these two adjec- 
tives the first point lying at one extreme, the last at the 
other extreme. For example the teacher asks his students: 

YOUR VIEW QE^GLOBAL EDUCATION 



1. easy difficult 



2 . interesting dull 



3. important trivial 



4. interdisciplinary simplistic 



5. fun . drudgery 



6. challenging boring 



6 Ibid. , p. 115. 
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. teacher then assigns a value from one to seven for each 
t between the extremes. Averaging the scores for each 
he can find a mean for each adjective set. These can be 
pared with the results for other views and with the results 
/ the same views at different times. These results can be 
aphically demonstrated by constructing profiles of the stu- 
mt responses: 

)UR VIEW OF GLOBAL EDUCATION 



difficult 



* dull 



trivial 



s impl is tic 



drudgery 



boring 



post- test 



- Likert Scale: 

The Likert Scale, too, is used to measure attitudes. The evalua- 
tor draws up a series of statements that will Indicate attitudes 
relevant to global studies: 

- Poor people are lazy 

- Indians should have fewer babies 

- Western former-colonial countries are to blame for 
the difficulties of the developing nations. 

The students respond to these statements on^a five point con- 
tinuum: strongly agree, agree, uncertain, disagree, strongly 
disagree. The statements should be meaningful, interesting, 
and as clear as possible to the students. Unsuccessful use of 
the Likert Scale technique is usually due to the failure of 
the statements to arouse the students' interest or 'to students 
skipping statements because *:hey are uncertain of their meaning. 7 

Using the students' responses, a class mean can again be estab- 
lished for each statement, using the same methods as in the last 
two examples. These means can be useful in demonstrating changes 
during the course. 



7 Ibid . , p. 120. 



1. 
2. 
3. 
4. 
5. 
6. 



easy 

interesting 
important 



interdisciplinary j£ 

i 

fun ~T 



challenging 
pre -test 
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SUMMARY 

These techniques may be used effectively for both summative 
(long-range) and formative (short-range) evaluation. The use 
made^will depend on the teacher and on his individual evalua- 
tion case. 

Here, again, emphasis is needed to stress the importance of 
making the students feel part of the testing process, con- 
vinced that they are not being critically judged and may, 
therefore, be candid in their thoughts and expressions. One 
way to achieve this is by insuring anonymity to the students 
at times. A number or letter code may be set up where the 
teacher does not necessarily identify the student with his 
answers. An essential part of the evaluation process is the 
reactions of the students to the course, its content, teaching, 
and what they feel they have gotten from it. A good technique 
for getting students used to expressing such opinions, is for 
them to keep a journal of their experiences and reactions to the 
course during the time that it is taught. 

Testing and other evaluation techniques are essential tools 
for the teacher/evaluator to use in compiling information to 
improve instructional technique. But this information can- 
not stand by itself. As such, it can only identify trends and 
assumptions. It must be combined with what the teacher knows 
to be the special characteristics of his students. Once these 
have been weighed, he is then in the position to interpret the 
evaluation results in the context of his particular class and 
course objectives. These statements are backed up by individual 
cases of evidence that the* teacher has compiled during the eval- 
uation. The strength of his results cannot rest on numbers, 
but on case in point. This he presents in the form of an ana- 
lytical summary of what he has learned with accompanying propos- 
als for applying this knowledge to future plans. 
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APPENDIX A 



The 'Logical Framework 1 



This appendix describes the construction and use of the 
'logical framework 1 , as adapted from the model used by 
the Agency for International Development of the U.S. 
Department of State. In such a discussion, it is useful 
to first repeat the purpose of such a framework in the 
teacher's undertaking of the evaluation process (from 
page 5) : 

define the goals of his course 

- outline the overall testing procedure and 
the purpose of each project within that pro- 
cedure 

- 'plan for methods and materials (input) and 

for expectation of achieved learning and 
under s tanding (output ) 

- specify evidence* (indicator s) chosen to 
verify output 

- state k assumptions about students, methods, 
materials, and the teacher, that are made 
*in choosing indicators and planning for 

output . 

, Although the design may change according to the individual 
teacher's specific needs and challenges, the basic logical 
framework matrix "looks like the grid in Chart 1 (page 17). 
The purpose of this framework for analysis is to set up, 
for the evaluator — and for those who would s\.udy his re- 
sults — a system that will**: 

- define, in measurable or objectively veri- 
fiable^ terms , project inputs, outputs, pro- 
gram goals, and objectives to reach those 
goals 

- establish the indicators which will permit 
subsequent measurement or verification of 

, achievement* of the defined outputs, objec- 
tives, and goals 



5 Agency for International Development, The Logical Framework - 
Modifications Based on Experience (Washington, D.C. : State 
Department , 1973) , p. 1 . ~~ 
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- identify external influences and factors 
(assumptions) which will affect linkages 
between inputs , outputs , obj ectives , and 
goals . 

A simple way to explain this framework is to work through 
a simple sample situation. Chart 2 (page 18) is an example 
of the type of framework that could be set up for a typical 
course in global studies . 

From this information, the evaluator can draw hypotheses 
about the causative (means/end) 1 inkages between the inputs, 
outputs, objectives, and goals. 9 The evaluation process 
checks these hypotheses for the overall course, at the 
same time monitoring progress from the level of input to 
that of output, then to objectives, and finally to" the 
goal itself. The diagram in Figure 1 (page 19) suggests 
basic questions to pursue in this process. 

Using the model course outline in Chart 2, the answers 
to the linkage evaluation questions in Figure 1 might be: 

Q. What types of inputs produce the best 
results? How might better results be 
obtained? 

A. Case studies increase depth of under- 
standing by taking the concepts of 
global development and tying them down 
to "real world" instances. Presenting 
two case studies at once an area a 
and an issue, for example would 
strengthen the student f s ability to 
apply knowledge . 

Q . What are the key elements that assured 
success thus far in achieving goal ob- 
j ectives? 

A. 'Basic understanding of economic prin- 
ciples has greatly increased the depth 
of the students 1 understanding of pre- 
viously learned history and current 
events. More study of government theory 
and institutions in combination with 
economics would result in a better under- 
standing of the relatedness of factors of 
development . 



9 Ibid. 



19 



- 16 



What are the key elements that assured 
success thus far in achieving the goal? 
Understanding the continued interrela- 
tedness of factors helps the student re- 
frane from superficial 1 right/wrong 1 
answers to complex global problems. A 
case study of his own community as a 
developing area would increase the 
'global 1 perspective of the student 1 s 
image of himself, his family, his- 
community, state and nation, under- 
standing that 'different* does not neces- 
sarily mean T wrong T . 
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CHART 1: LOGICAL FRAMEWORK 



NARRATIVE SUMMARY 


INDICATORS 


EVIDENCE 


ASSUMPTIONS 


Course Goal: 

[what is the general 
purpose of the 
course?] 


[what are the measures 
for goal achievement? ] 


[what are the means for 
verifying the indica- 
tors of achieving the 
course -goal ? ] 


[what assumptions were 
made in determining 
' what evidence would 
vaiiQaie tne lnaicd 
tors of goal achieve- 
ment?] 


Goal Objectives: 

[what combination of 
basic understanding 
will result in 
meeting course goals?] 


[what conditions will, 
indicate achievement 
of goal objectives?] 


[what are the means for 
verifying the indica- 
lots or dcnicving tne 
goal obj ect ives ?] 


[what assumptions were 
made in determining 
what evidence would 

Aral i a t* p t*Vio H n i r a - 
Vdl IQd Lc LJlc J.J1UJ.L.C1 

tors of achieving 
the goal objectives?] 


Course Output: 

[what achieved learning 
is expected?] 


[what learning is 
deemed necessary for 
achievement of goal 
obj ective?] 


[what are the means 
for verifying the 
indicators of 
acnievmg tne aesirea 
course output?] 


[what assumptions were 
made in determining 
what evidence would 

Vdl IQd Lc dL Jl J. c V clllcll L 

of course output?] 


Course Input : 

[what methods and 
materials are being 
us e a ( j 


[what type and quantity 
of information is made 
available by the^e 

III c L J 1U Li o dilLL UL^L-CI -L ex ± o 

to achieve the desired 
output?] 


[what are the means for 
verifying the indica- 
tors of the worth of 
thp mpfhnrl^ and 
materials cited?] 


[what assumptions were 
made in determining 
what evidence would 
validate the worth 
of course input?] 
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CHART 2: LOGICAL FRAMEWORK 
(An example only, not representative of an actual course ) 



NARRATIVE SUMMARY 


INDICATORS 


EVIDENCE 


ASSUMPTIONS 


Course Goal: 

- to broaden the stu- 
dent ! s perspectives to a 
more global outlook to 
prepare him for the in- 
terdependent world he 
,will meet, live, and 
work in as an adult. 


- when discussing global 
issues, the student ties 
each with other issues 
in a continuum 

- the student avoids 
'we/they 1 analyses 

- the student avoids 

? right /wrong 1 analyses 


Interviews 

Student evaluations of 
course 

Obj ective testing 
Research Droierts 


- the student is relaxed 
enough to make candid 
observations 

- the tearher is candid in 
his analysis of the evi- 
dence colleqted 


Goal Objectives: 

- to -recognize factors 
of development 

- to recognize the 
int err elatedness of 
these factors 


- when discussing global 
issues, the student in- 
cludes economic , social * 
historic , cultural 
factors 

- when discussing one of 
the above factors, he 
does so in the context 
of the others 


Research- proj ect s 
Debates 

Obj edtive testing 
Essay questions 
In- class discussion 


The student has access to 
the library 

The student is used to ex- 
pressing himself verbally 

The student is used to ex- 
pressing himself in 
writing 


Course Output: 
i- a basic understanding 
of Economics 

- a basic grasp of re- 
search techniques 

- a basic grasp of 
statistics 

- a basic understanding 
of Geography 


Understanding principles 
of supply and demand 
Understanding tariffs 
and price supports 
Ability to read charts 
and graphs 

Geographic knowledge of 
continents , climate , 
resources , topography 


Obj ective testing 
Plans for research 
Group reports 
In-class discussion 


The student has access 

to the library 

The student has access to 

the course inputs 

The student can express 

himself verbally 

The student is provided 

in-class and out-of-class 

time to meet and work in 

groups 


Course Input : [List] 

Texts, other materials 

Films 

Maps 

Community speakers with 
development experience 
Teacher: experience and 
academic background 


- basic information on 
development concepts 

- basic geographic in- 
formation 

- basic economic infor- 
m a t i on 

- an understanding that 
" f global'- ! real world 1 

- prepare student to 
support and trade ideas 


: Films and materials sug- 
gested by a*x organiza- 
tion specializing in 
global development edu- 
cation 

H TOi in P pnnrt <; ' all m pm ~ 

bers responsible for var- 
ious parts, then made in- 
to a whole presentation 
Obj ective testing 


- the student has a Math 
background adequate for 
discerning economic rela- 
tionships 

- the advising organiza- 

LJ.UI1 fldb SULLcSSIUl cA 

per ience in its work 

- the teacher knows his 
students well enough to 
make up workable groups 
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FIGURE 1 



LINKAGES 



Course 
goal 




If obj ectives , 
then goal 



Goal 

obj ectives 




If outputs, 
then objectives" 



Course 
outputs 



START 
HERE 




If inputs, 
then outputs" 



Course 
inputs 



EVALUATION QUESTIONS 



_Why is this goal valid 
to the purpose of the 
school and community? 
[Figure 2] 



What are the key ele- 
ments that have assured 
_success thus far? What 
can be done to increase 
the chance of success? 



What basic understandings 
"are sought? [Figure 2] 



What are the key ele- 
ments that have assured 
success thus far? What 
can be done to increase 
the chance of success? 



JVhat basic knowledge 
"is aimed at? [Figure 2] 



What types of inputs * 
produce the best re- 
sults? How ma/ better 
results be obtained? 



JVhat methods and materials 
"are employed? [Figure 2] 



START WITH 'INPUTS' AND MOVE 
UPWARD, FOLLOWING ARROWS 




APPENDIX B 



Testing Models 



This appendix contains examples of the written testing tech- 
niques cited in the preceding paper. The SURVEY QUESTIONNAIRE 
ON GLOBAL DEVELOPMENT is taken or adapted from inventory — 
Global Poverty/Development", an exercise designed by World 
Neighbors; Kathleen Desmond, Development : Bridgeto Peace 
(Washington, D.C.: American Freedom From Hunger Foundation, 
1970); Church Women United Survey designed by Chris Cowap ; and 
various Overseas Development Council publications. It can be 
found in Focusing on Global Poverty and Development: A Resource 
Book for Educators , by Jayne M. Wood, published by the Overseas 
Development Council in Washington, D.C. Many global studies 
teachers have found it a useful tool for evaluation. The ques- 
tions found for the Likert Scale are also part of this excellent 
resource book. The Semantic Differential comes from Handbook 
of Simulation Gaming in Social Education (Part 1*: Textbook) , by 
Ron Stadsklev and available from the Institute of Higher Educa- 
tion Research and Services of the University of Alabama at 
Univers ity , Alabama . 

These examples should be viewed as models - 7 rath.27 than as dic- 
tates for testing. Once again, each teacher will want to use 
them as best befits his individual case.' 
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SURVEY QUESTIONNAIRE ON GLOBAL DEVELOPMENT 



The purpose of this exercise is to get an idea of your under- 
standing of and feelings about poverty and development. You 
are not expected to know the answers to every question. Please 
write your answer(s) in the blank at the right. 



What percent of the world 1 s population lives 
in developing countries? 



a. less than 10% c. about 25% e. about 50% 

b. about 75o d. about 90% 

Approximately what percent of the world's 

population does the United States have? 

a. 3% c. 6% e. 9% 

b. 12% d. 15% 

Approximately what percent of the world's 

wealth does the United States have? 

a. about 15% c. about 25% e. about 35% 

b. about 50% d. about 65% 

Which three continents contain the majority 

of developing countries? 

a. Asia c. Africa e. Australia 

b. North America d. Latin America f. Europe 

Which three of the following countries have 

the highest rate of population growth? 

a . India d . Japan g . Mexico 

b. Honduras e. Nigeria h. China (Mainland) 

c. Saudi Arabia f. U.S. i. Portugal 

Which three of the f ol lowing countries have 

the lowest rate of population growth? 

a . India d . Japan g . Mexico 

b. Honduras e. Nigeria h. China (Mainland) 

c. Saudia Arabia f. U.S. i. Portugal 
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7. Which three countries in the following list 

have the largest populations? 

a. Pakistan § Bangladesh d. Germany 

b. Venezuela e. Mexico 

c. Nigeria f. Japan 

8. In which region do you think one would find 

the lowest rate of literacy? 

a. Asia c. Latin America e. Africa 

b. Oceania d. Europe f. North America 

9. What is the "green revolution"? 



a. a guerilla war 

b. a back-to- the-land movement 

c. a breakthrough in food production 

d. a victory for new kinds of pesticides 

e. a new method of cutting back tropical overgrowth- 

10. Which five countries do you believe have the _ 
highest standard of living based on per capita 
income or GNP per capita? (This figure is ob- 
tained by dividing the total output of a 

country — total GNP by its total population,) 

a. Kenya e. South Africa h. India 

b. Czechoslovakia f. Mongolia i. Libya 

c. Ecuador g. Argentina j. Denmark 

d. South Vietnam 

11. Which five countries do you believe have the _ 
lowest standard of living based on per capita 
income or GNP per capita? 

a. Kenya e. South Africa h. India 

b . Czechoslovakia f . Mongol ia i . Libya 

c. Ecuador g. Argentina j . Denmark 

d. South Vietnam 

12. Which item do you think receives the biggest 
"chunk" of the U.S. budget? 

a. foreign aid d. educational programs 

b . mil itary e . government operations 

c. health programs f. welfare programs 

13. What percent of the U.S. GNP do you think _ 
goes to foreign assistance? 

a. about 25% c. about 15% e. about 10% 

b. about 5% d. under 1% 

ERIC ZS 
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14. Do you think that U.S. aid to developing 
countries as a percent of GNP is: 

a . more than that of any other developed country 

b. more than most developed countries 

c. about, the same as other developed countries 

d. less than many other developed countries 

15. Using the percent - of - GNP formula , where does 
the U.S. rank among all other countries in 
money given to the United Nations? 

a. 1st c. 12th e. 35th 

b. 52nd d. 65th 



16. In 1958, a Central African earned enough money, 
from selling us 200 pounds of cotton, to buy 
four blankets. How many blankets could he buy 
today for the money earned from selling the 
same amount of cotton? 

a. 1 b. 4 c. 8 d. 12 

17. Unemployment in the developing world ranges 
from of the labor force. 

a. 20-25% c. 5-8% d. 1-3% 

b. 13-15% 

18. The average protein intake of each person in 
the U.S. is about 96 grams per day, in India 
it is : 

a. about the same c. 3/4 as much 

b. 1/2 as much d. 1/3 as much 

19. If current population growth rates are 
maintained, the number of people in the world 
will by the year 2000. 

a . remain about the same due to an increase 
in the practice of birth control 

b. increase by 15% 

c. double itself 

d. be increased by 50% 

20. The people of the United States make up less 
than 7% of the world f s population, yet they 
use about of the world f s non-renewable 
resources (such as aluminium, iron, and natural 
gas). 



a . 
b. 



40% 
65% 



c. 75% 

d. 90% 
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ANSWERS TO SURVEY QUESTIONNAIRE 

1. b. about 75 percent 

2. c. 6 percent 

3. e. about 35 percent 

4. a. Asia c. Africa d . Latin America 

5. b. Honduras e. Nigeria g. Mexico 

6. d. Japan f. U.S. i. Portugal 

7. a. Pakistan and Bangladesh d. Germany £. Japan 

8. e. Africa 

9. c. breakthrough in food, product ion 

10. b. Czechoslovakia e. South Africa g. Argentina 
i . Libya j . Denmark 

(Remember: per capita GNP is an average figure and doesn't 
indicate distribution of wealth. Some of the poorest 
countries have very rich sectors in their population, 
and this will pull up the average figure.) 

11. a. Kenya c. Ecuador d. South Vietnam 
f. Mongolia h. India 

12. b. military 

13. d. under 1 percent 

14. d. less than many other developed countries 

15. b. 52nd 

16. - a. 1 

17. a. 20-25% 

18. b. one half as much 

19. c. double itself 

20. a. 40 percent 
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INTEREST SCALE 



Directions: You are asked to rate the following topics in 

terms of how interesting you think they would be 
to study: 

A. dull 

B. generally uninteresting 

C . generally interesting 

D. very interesting 

There are no right answers; only your opinions are sought. 



1. 


Politics 


2. 


World Food Situation 


3. 


Population Growth 


4. 


National Customs 


5. 


Tariff Policies 


6 . 


Foreign Aid 


7 . 


National Monuments 


8. 


Comparative Economic Systems 


9. 


National Costumes 


10. 


Poverty 
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SEMANTIC DIFFERENTIAL 



Directions: In this test we want to find out how you describe 
different things. As before, there are no right 
and wrong answers. You will find a word printed 
like this : 

GEOGRAPHY 

Look at the word; get an idea of it in you own 
mind. Below the word you will find a number of 
words which describe geography. These words are 
'put in pairs that have opposite meaning. Between 
the meanings are seven spaces. You are asked to 
fill in the space that you feel best describes 
geography. The first has been done as an example. 



GEOGRAPHY 



Logical 

Meaningful 

Hard 

Clear 

Important 

Useful 

Analytical 



Important for 
the Future ~ 

Interesting 



Illogical 

Meaningless 

Easy 

Unclear 

Unimportant 

Useless 

Descriptive 



Not important for 
the Future 

Uninteresting 



Precise 



Vague 
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LIKERT SCALE 



Directions: Each of the following statements is followed by 

five responses. Indicate how you feel about each 
statement by circling one of the responses: 

SA if you- "strongly agree" 

A if you simply "agree" 

UN if you are "uncertain" 

DA if you simply "disagree" 

SDA if you "strongly disagree" 

There are no right answers; only your opinions are sought. 



1. People are poor because they are lazy. SA A UN DA SDA 

2. Most Americans have an accurate idea of SA A UN DA SDA 
how Africans live. 

3. If poor nations would adopt the U.S. SA A UN DA SDA 
economic and political systems of 

capitalism and democracy, they would 
develop more rapidly. 

4. The U.S. government should spend more SA A UN DA SDA 
to help developing countries. 

5. Poverty in the U.S. and poverty in the SA A UN DA SDA 
developing countries of the world are 

not really related in any way. 

6. If developing countries would adopt SA A UN DA SDA 
a Communist Chinese economic and 

political system, they would develop 
more rapidly . 

7. Discrimination and oppression cause SA A UN DA SDA 
poverty around the world. 



GLOSSARY 



The purpose of this brief Glossary is to give the author's 
usage of terms employed in the foregoing paper, which other- 
wise might cause confusion for the reader. 



AFFECTIVE DOMAIN: That area of thought derived from inter- 
pretation through one's values and feelings, rather than 
through perception, reasoning, or intuition. 

ASSUMPTIONS: External influences and factors affecting the 

the links from course input through achievement of course 
goals . 

COGNITIVE DOMAIN: That area of thought derived from perception 
or reasoning. Knowledge . 

FORMATIVE TESTING: 1 Short-range T evaluation. Used during the 
teaching process, rather than at the end, its purpose is 
to observe the parts, rather than the whole of curriculum 
design and adaptation, teaching, or learning. 

GOALS: Within the context of the evaluation process, the goals 
of a course describe the main purpose of it. 

INDICATORS: In the evaluation procedure, the various factors 
.that permit verification of achievement of outputs, ob- 
jectives, and goals. 

INPUT: Within the context of the evaluation process, the 

teaching methods and materials -- texts, films, simulations, 
speakers, etc. of a course. 

INTEREST SCALE: A testing technique to discern student atti- 
tudes through their degree of interest in certain sub- 
ject or, topic areas. 

LIKERT SCALE: A testing technique to discern student attitudes 
through the degree of validity they assign to certain 
j udgmental statements . 

LOGICAL FRAMEWORK: A method for setting up the design for an 

evaluation procedure that provides the teacher with a clear 
understanding of that process. Adapted from the model 
used by the Agency of International Development of the De- 
partment of State. 
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OBJECTIVES: Within the context of the evaluation process, the 
objectives of a course describe the various factors that 
make up the course goals. 

OUTPUT: Within the context of the evaluation process, the 

learning and understanding achieved through the course 
input that will lead to fulfilling the course ob j ectives . 

SEMANTIC DIFFERENTIAL: A testing technique to discern student 
attitudes through the degree of value they assign certain 
ideas or objects. 

SUMMATIVE TESTING: The purpose of this long—range testing is 

to observe overall results of an entire course, or substan- 
tial parts of it. 
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